{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "ea63cd0b-9873-4afd-815c-35a056bc0e37",
   "metadata": {},
   "source": [
    "# Order Latency Data\n",
    "\n",
    "To obtain more realistic backtesting results, accounting for latencies is crucial. Therefore, it's important to collect both feed data and order data with timestamps to measure your order latency. The best approach is to gather your own order latencies. You can collect order latency based on your live trading or by regularly submitting orders at a price that cannot be filled and then canceling them for recording purposes. However, if you don't have access to them or want to establish a target, you will need to artificially generate order latency. You can model this latency based on factors such as feed latency, trade volume, and the number of events. In this guide, we will demonstrate a simple method to generate order latency from feed latency using a multiplier and offset for adjustment."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ce73ae54-2e57-489e-919e-d1f9271f4461",
   "metadata": {},
   "source": [
    "First, loads the feed data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "ab04f05e-259b-4a68-b701-1df3f2c19a82",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([(3758096386, 1580515202342000000, 1580515202497052000, 9364.51, 1.197, 0, 0, 0.),\n",
       "       (3758096386, 1580515202342000000, 1580515202497346000, 9365.67, 0.02 , 0, 0, 0.),\n",
       "       (3758096386, 1580515202342000000, 1580515202497352000, 9365.86, 0.01 , 0, 0, 0.),\n",
       "       ...,\n",
       "       (3489660929, 1580601599836000000, 1580601599962961000, 9351.47, 3.914, 0, 0, 0.),\n",
       "       (3489660929, 1580601599836000000, 1580601599963461000, 9397.78, 0.1  , 0, 0, 0.),\n",
       "       (3489660929, 1580601599848000000, 1580601599973647000, 9348.14, 3.98 , 0, 0, 0.)],\n",
       "      dtype=[('ev', '<i8'), ('exch_ts', '<i8'), ('local_ts', '<i8'), ('px', '<f8'), ('qty', '<f8'), ('order_id', '<u8'), ('ival', '<i8'), ('fval', '<f8')])"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import numpy as np\n",
    "\n",
    "data = np.load('btcusdt_20200201.npz')['data']\n",
    "data"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ef2deb8c-4cd3-44ad-a639-b42f126be63a",
   "metadata": {},
   "source": [
    "For easy manipulation, converts it into a DataFrame."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "bb32210a-34b6-489d-b32d-0522ce9daa97",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div><style>\n",
       ".dataframe > thead > tr,\n",
       ".dataframe > tbody > tr {\n",
       "  text-align: right;\n",
       "  white-space: pre-wrap;\n",
       "}\n",
       "</style>\n",
       "<small>shape: (27_532_602, 8)</small><table border=\"1\" class=\"dataframe\"><thead><tr><th>ev</th><th>exch_ts</th><th>local_ts</th><th>px</th><th>qty</th><th>order_id</th><th>ival</th><th>fval</th></tr><tr><td>i64</td><td>i64</td><td>i64</td><td>f64</td><td>f64</td><td>u64</td><td>i64</td><td>f64</td></tr></thead><tbody><tr><td>3758096386</td><td>1580515202342000000</td><td>1580515202497052000</td><td>9364.51</td><td>1.197</td><td>0</td><td>0</td><td>0.0</td></tr><tr><td>3758096386</td><td>1580515202342000000</td><td>1580515202497346000</td><td>9365.67</td><td>0.02</td><td>0</td><td>0</td><td>0.0</td></tr><tr><td>3758096386</td><td>1580515202342000000</td><td>1580515202497352000</td><td>9365.86</td><td>0.01</td><td>0</td><td>0</td><td>0.0</td></tr><tr><td>3758096386</td><td>1580515202342000000</td><td>1580515202497357000</td><td>9366.36</td><td>0.002</td><td>0</td><td>0</td><td>0.0</td></tr><tr><td>3758096386</td><td>1580515202342000000</td><td>1580515202497363000</td><td>9366.36</td><td>0.003</td><td>0</td><td>0</td><td>0.0</td></tr><tr><td>&hellip;</td><td>&hellip;</td><td>&hellip;</td><td>&hellip;</td><td>&hellip;</td><td>&hellip;</td><td>&hellip;</td><td>&hellip;</td></tr><tr><td>3489660929</td><td>1580601599812000000</td><td>1580601599944404000</td><td>9397.79</td><td>0.0</td><td>0</td><td>0</td><td>0.0</td></tr><tr><td>3489660929</td><td>1580601599826000000</td><td>1580601599952176000</td><td>9354.8</td><td>4.07</td><td>0</td><td>0</td><td>0.0</td></tr><tr><td>3489660929</td><td>1580601599836000000</td><td>1580601599962961000</td><td>9351.47</td><td>3.914</td><td>0</td><td>0</td><td>0.0</td></tr><tr><td>3489660929</td><td>1580601599836000000</td><td>1580601599963461000</td><td>9397.78</td><td>0.1</td><td>0</td><td>0</td><td>0.0</td></tr><tr><td>3489660929</td><td>1580601599848000000</td><td>1580601599973647000</td><td>9348.14</td><td>3.98</td><td>0</td><td>0</td><td>0.0</td></tr></tbody></table></div>"
      ],
      "text/plain": [
       "shape: (27_532_602, 8)\n",
       "┌────────────┬─────────────────────┬────────────────────┬─────────┬───────┬──────────┬──────┬──────┐\n",
       "│ ev         ┆ exch_ts             ┆ local_ts           ┆ px      ┆ qty   ┆ order_id ┆ ival ┆ fval │\n",
       "│ ---        ┆ ---                 ┆ ---                ┆ ---     ┆ ---   ┆ ---      ┆ ---  ┆ ---  │\n",
       "│ i64        ┆ i64                 ┆ i64                ┆ f64     ┆ f64   ┆ u64      ┆ i64  ┆ f64  │\n",
       "╞════════════╪═════════════════════╪════════════════════╪═════════╪═══════╪══════════╪══════╪══════╡\n",
       "│ 3758096386 ┆ 1580515202342000000 ┆ 158051520249705200 ┆ 9364.51 ┆ 1.197 ┆ 0        ┆ 0    ┆ 0.0  │\n",
       "│            ┆                     ┆ 0                  ┆         ┆       ┆          ┆      ┆      │\n",
       "│ 3758096386 ┆ 1580515202342000000 ┆ 158051520249734600 ┆ 9365.67 ┆ 0.02  ┆ 0        ┆ 0    ┆ 0.0  │\n",
       "│            ┆                     ┆ 0                  ┆         ┆       ┆          ┆      ┆      │\n",
       "│ 3758096386 ┆ 1580515202342000000 ┆ 158051520249735200 ┆ 9365.86 ┆ 0.01  ┆ 0        ┆ 0    ┆ 0.0  │\n",
       "│            ┆                     ┆ 0                  ┆         ┆       ┆          ┆      ┆      │\n",
       "│ 3758096386 ┆ 1580515202342000000 ┆ 158051520249735700 ┆ 9366.36 ┆ 0.002 ┆ 0        ┆ 0    ┆ 0.0  │\n",
       "│            ┆                     ┆ 0                  ┆         ┆       ┆          ┆      ┆      │\n",
       "│ 3758096386 ┆ 1580515202342000000 ┆ 158051520249736300 ┆ 9366.36 ┆ 0.003 ┆ 0        ┆ 0    ┆ 0.0  │\n",
       "│            ┆                     ┆ 0                  ┆         ┆       ┆          ┆      ┆      │\n",
       "│ …          ┆ …                   ┆ …                  ┆ …       ┆ …     ┆ …        ┆ …    ┆ …    │\n",
       "│ 3489660929 ┆ 1580601599812000000 ┆ 158060159994440400 ┆ 9397.79 ┆ 0.0   ┆ 0        ┆ 0    ┆ 0.0  │\n",
       "│            ┆                     ┆ 0                  ┆         ┆       ┆          ┆      ┆      │\n",
       "│ 3489660929 ┆ 1580601599826000000 ┆ 158060159995217600 ┆ 9354.8  ┆ 4.07  ┆ 0        ┆ 0    ┆ 0.0  │\n",
       "│            ┆                     ┆ 0                  ┆         ┆       ┆          ┆      ┆      │\n",
       "│ 3489660929 ┆ 1580601599836000000 ┆ 158060159996296100 ┆ 9351.47 ┆ 3.914 ┆ 0        ┆ 0    ┆ 0.0  │\n",
       "│            ┆                     ┆ 0                  ┆         ┆       ┆          ┆      ┆      │\n",
       "│ 3489660929 ┆ 1580601599836000000 ┆ 158060159996346100 ┆ 9397.78 ┆ 0.1   ┆ 0        ┆ 0    ┆ 0.0  │\n",
       "│            ┆                     ┆ 0                  ┆         ┆       ┆          ┆      ┆      │\n",
       "│ 3489660929 ┆ 1580601599848000000 ┆ 158060159997364700 ┆ 9348.14 ┆ 3.98  ┆ 0        ┆ 0    ┆ 0.0  │\n",
       "│            ┆                     ┆ 0                  ┆         ┆       ┆          ┆      ┆      │\n",
       "└────────────┴─────────────────────┴────────────────────┴─────────┴───────┴──────────┴──────┴──────┘"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import polars as pl\n",
    "\n",
    "df = pl.DataFrame(data)\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0a2f5db2-b6ab-420d-b5f9-0ca8e123607b",
   "metadata": {},
   "source": [
    "Selects only the events that have both a valid exchange timestamp and a valid local timestamp to get feed latency."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "349335b0-50a4-4e3a-a721-c55388d71b01",
   "metadata": {},
   "outputs": [],
   "source": [
    "from hftbacktest import EXCH_EVENT, LOCAL_EVENT\n",
    "\n",
    "df = df.filter((pl.col('ev') & EXCH_EVENT == EXCH_EVENT) & (pl.col('ev') & LOCAL_EVENT == LOCAL_EVENT))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e641eb14-5370-4cc3-bf52-e940d94745a1",
   "metadata": {},
   "source": [
    "Reduces the number of rows by resampling to approximately 1-second intervals."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "577ad180-42ad-46f8-9723-1b2c9b14d9c2",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div><style>\n",
       ".dataframe > thead > tr,\n",
       ".dataframe > tbody > tr {\n",
       "  text-align: right;\n",
       "  white-space: pre-wrap;\n",
       "}\n",
       "</style>\n",
       "<small>shape: (86_394, 2)</small><table border=\"1\" class=\"dataframe\"><thead><tr><th>exch_ts</th><th>local_ts</th></tr><tr><td>i64</td><td>i64</td></tr></thead><tbody><tr><td>1580515202843000000</td><td>1580515202979365000</td></tr><tr><td>1580515203551000000</td><td>1580515203943566000</td></tr><tr><td>1580515203789000000</td><td>1580515204875639000</td></tr><tr><td>1580515204127000000</td><td>1580515205962135000</td></tr><tr><td>1580515204738000000</td><td>1580515206983780000</td></tr><tr><td>&hellip;</td><td>&hellip;</td></tr><tr><td>1580601595869000000</td><td>1580601595997115000</td></tr><tr><td>1580601596865000000</td><td>1580601596994060000</td></tr><tr><td>1580601597864000000</td><td>1580601597987786000</td></tr><tr><td>1580601598870000000</td><td>1580601598997068000</td></tr><tr><td>1580601599848000000</td><td>1580601599973647000</td></tr></tbody></table></div>"
      ],
      "text/plain": [
       "shape: (86_394, 2)\n",
       "┌─────────────────────┬─────────────────────┐\n",
       "│ exch_ts             ┆ local_ts            │\n",
       "│ ---                 ┆ ---                 │\n",
       "│ i64                 ┆ i64                 │\n",
       "╞═════════════════════╪═════════════════════╡\n",
       "│ 1580515202843000000 ┆ 1580515202979365000 │\n",
       "│ 1580515203551000000 ┆ 1580515203943566000 │\n",
       "│ 1580515203789000000 ┆ 1580515204875639000 │\n",
       "│ 1580515204127000000 ┆ 1580515205962135000 │\n",
       "│ 1580515204738000000 ┆ 1580515206983780000 │\n",
       "│ …                   ┆ …                   │\n",
       "│ 1580601595869000000 ┆ 1580601595997115000 │\n",
       "│ 1580601596865000000 ┆ 1580601596994060000 │\n",
       "│ 1580601597864000000 ┆ 1580601597987786000 │\n",
       "│ 1580601598870000000 ┆ 1580601598997068000 │\n",
       "│ 1580601599848000000 ┆ 1580601599973647000 │\n",
       "└─────────────────────┴─────────────────────┘"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = df.with_columns(\n",
    "    pl.col('local_ts').alias('ts')\n",
    ").group_by_dynamic(\n",
    "    'ts', every='1000000000i'\n",
    ").agg(\n",
    "    pl.col('exch_ts').last(),\n",
    "    pl.col('local_ts').last()\n",
    ").drop('ts')\n",
    "\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7834be83-788e-48ae-92e9-0f01bda90cd5",
   "metadata": {},
   "source": [
    "Converts back to the structured NumPy array."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "d69e7709-d947-4e62-b5a4-c8f1e623de2a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([(1580515202843000000, 1580515202979365000),\n",
       "       (1580515203551000000, 1580515203943566000),\n",
       "       (1580515203789000000, 1580515204875639000), ...,\n",
       "       (1580601597864000000, 1580601597987786000),\n",
       "       (1580601598870000000, 1580601598997068000),\n",
       "       (1580601599848000000, 1580601599973647000)],\n",
       "      dtype=[('exch_ts', '<i8'), ('local_ts', '<i8')])"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data = df.to_numpy(structured=True)\n",
    "data"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4d67d41d-2ef7-4267-80e9-9bc7bbd5a271",
   "metadata": {},
   "source": [
    "Generates order latency. Order latency consists of two components: the latency until the order request reaches the exchange's matching engine and the latency until the response arrives backto the localy. Order latency is not the same as feed latency and does not need to be proportional to feed latency. However, for simplicity, we model order latency to be proportional to feed latency using a multiplier and offset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "1de7a129-5856-473e-8453-effb1f3884cd",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([(1580515202979365000, 1580515203524825000, 1580515203933920000, 0),\n",
       "       (1580515203943566000, 1580515205513830000, 1580515206691528000, 0),\n",
       "       (1580515204875639000, 1580515209222195000, 1580515212482112000, 0),\n",
       "       ...,\n",
       "       (1580601597987786000, 1580601598482930000, 1580601598854288000, 0),\n",
       "       (1580601598997068000, 1580601599505340000, 1580601599886544000, 0),\n",
       "       (1580601599973647000, 1580601600476235000, 1580601600853176000, 0)],\n",
       "      dtype=[('req_ts', '<i8'), ('exch_ts', '<i8'), ('resp_ts', '<i8'), ('_padding', '<i8')])"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mul_entry = 4\n",
    "offset_entry = 0\n",
    "\n",
    "mul_resp = 3\n",
    "offset_resp = 0\n",
    "\n",
    "order_latency = np.zeros(len(data), dtype=[('req_ts', 'i8'), ('exch_ts', 'i8'), ('resp_ts', 'i8'), ('_padding', 'i8')])\n",
    "for i, (exch_ts, local_ts) in enumerate(data):\n",
    "    feed_latency = local_ts - exch_ts\n",
    "    order_entry_latency = mul_entry * feed_latency + offset_entry\n",
    "    order_resp_latency = mul_resp * feed_latency + offset_resp\n",
    "\n",
    "    req_ts = local_ts\n",
    "    order_exch_ts = req_ts + order_entry_latency\n",
    "    resp_ts = order_exch_ts + order_resp_latency\n",
    "    \n",
    "    order_latency[i] = (req_ts, order_exch_ts, resp_ts, 0)\n",
    "    \n",
    "order_latency"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "cea9998d-ed9f-4ac0-aa0e-ecd791e717a5",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div><style>\n",
       ".dataframe > thead > tr,\n",
       ".dataframe > tbody > tr {\n",
       "  text-align: right;\n",
       "  white-space: pre-wrap;\n",
       "}\n",
       "</style>\n",
       "<small>shape: (86_394, 4)</small><table border=\"1\" class=\"dataframe\"><thead><tr><th>req_ts</th><th>exch_ts</th><th>resp_ts</th><th>_padding</th></tr><tr><td>i64</td><td>i64</td><td>i64</td><td>i64</td></tr></thead><tbody><tr><td>1580515202979365000</td><td>1580515203524825000</td><td>1580515203933920000</td><td>0</td></tr><tr><td>1580515203943566000</td><td>1580515205513830000</td><td>1580515206691528000</td><td>0</td></tr><tr><td>1580515204875639000</td><td>1580515209222195000</td><td>1580515212482112000</td><td>0</td></tr><tr><td>1580515205962135000</td><td>1580515213302675000</td><td>1580515218808080000</td><td>0</td></tr><tr><td>1580515206983780000</td><td>1580515215966900000</td><td>1580515222704240000</td><td>0</td></tr><tr><td>&hellip;</td><td>&hellip;</td><td>&hellip;</td><td>&hellip;</td></tr><tr><td>1580601595997115000</td><td>1580601596509575000</td><td>1580601596893920000</td><td>0</td></tr><tr><td>1580601596994060000</td><td>1580601597510300000</td><td>1580601597897480000</td><td>0</td></tr><tr><td>1580601597987786000</td><td>1580601598482930000</td><td>1580601598854288000</td><td>0</td></tr><tr><td>1580601598997068000</td><td>1580601599505340000</td><td>1580601599886544000</td><td>0</td></tr><tr><td>1580601599973647000</td><td>1580601600476235000</td><td>1580601600853176000</td><td>0</td></tr></tbody></table></div>"
      ],
      "text/plain": [
       "shape: (86_394, 4)\n",
       "┌─────────────────────┬─────────────────────┬─────────────────────┬──────────┐\n",
       "│ req_ts              ┆ exch_ts             ┆ resp_ts             ┆ _padding │\n",
       "│ ---                 ┆ ---                 ┆ ---                 ┆ ---      │\n",
       "│ i64                 ┆ i64                 ┆ i64                 ┆ i64      │\n",
       "╞═════════════════════╪═════════════════════╪═════════════════════╪══════════╡\n",
       "│ 1580515202979365000 ┆ 1580515203524825000 ┆ 1580515203933920000 ┆ 0        │\n",
       "│ 1580515203943566000 ┆ 1580515205513830000 ┆ 1580515206691528000 ┆ 0        │\n",
       "│ 1580515204875639000 ┆ 1580515209222195000 ┆ 1580515212482112000 ┆ 0        │\n",
       "│ 1580515205962135000 ┆ 1580515213302675000 ┆ 1580515218808080000 ┆ 0        │\n",
       "│ 1580515206983780000 ┆ 1580515215966900000 ┆ 1580515222704240000 ┆ 0        │\n",
       "│ …                   ┆ …                   ┆ …                   ┆ …        │\n",
       "│ 1580601595997115000 ┆ 1580601596509575000 ┆ 1580601596893920000 ┆ 0        │\n",
       "│ 1580601596994060000 ┆ 1580601597510300000 ┆ 1580601597897480000 ┆ 0        │\n",
       "│ 1580601597987786000 ┆ 1580601598482930000 ┆ 1580601598854288000 ┆ 0        │\n",
       "│ 1580601598997068000 ┆ 1580601599505340000 ┆ 1580601599886544000 ┆ 0        │\n",
       "│ 1580601599973647000 ┆ 1580601600476235000 ┆ 1580601600853176000 ┆ 0        │\n",
       "└─────────────────────┴─────────────────────┴─────────────────────┴──────────┘"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_order_latency = pl.DataFrame(order_latency)\n",
    "df_order_latency"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0c0eb2a2-64dd-4d4a-93f6-0c52a4bdc142",
   "metadata": {},
   "source": [
    "Checks if latency has invalid negative values."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "542eace5-590e-4d30-a552-0cf0825885d5",
   "metadata": {},
   "outputs": [],
   "source": [
    "order_entry_latency = df_order_latency['exch_ts'] - df_order_latency['req_ts']\n",
    "order_resp_latency = df_order_latency['resp_ts'] - df_order_latency['exch_ts']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "a64aad08-888f-48e7-85cc-1719ba3333b6",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "(order_entry_latency <= 0).sum()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "7e770e94-4660-4b0b-827b-951a046d9529",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "(order_resp_latency <= 0).sum()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "18944b3a-747c-4c38-96e1-0ab6930e098c",
   "metadata": {},
   "source": [
    "Here, we wrap the entire process into a method with `njit` for increased speed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "15b080d5-9e5f-4124-a83f-813bb2ff9414",
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "from numba import njit\n",
    "import polars as pl\n",
    "from hftbacktest import LOCAL_EVENT, EXCH_EVENT\n",
    "\n",
    "@njit\n",
    "def generate_order_latency_nb(data, order_latency, mul_entry, offset_entry, mul_resp, offset_resp):\n",
    "    for i in range(len(data)):\n",
    "        exch_ts = data[i].exch_ts\n",
    "        local_ts = data[i].local_ts\n",
    "        feed_latency = local_ts - exch_ts\n",
    "        order_entry_latency = mul_entry * feed_latency + offset_entry\n",
    "        order_resp_latency = mul_resp * feed_latency + offset_resp\n",
    "\n",
    "        req_ts = local_ts\n",
    "        order_exch_ts = req_ts + order_entry_latency\n",
    "        resp_ts = order_exch_ts + order_resp_latency\n",
    "\n",
    "        order_latency[i].req_ts = req_ts\n",
    "        order_latency[i].exch_ts = order_exch_ts\n",
    "        order_latency[i].resp_ts = resp_ts\n",
    "\n",
    "def generate_order_latency(feed_file, output_file = None, mul_entry = 1, offset_entry = 0, mul_resp = 1, offset_resp = 0):\n",
    "    data = np.load(feed_file)['data']\n",
    "    df = pl.DataFrame(data)\n",
    "    \n",
    "    df = df.filter(\n",
    "        (pl.col('ev') & EXCH_EVENT == EXCH_EVENT) & (pl.col('ev') & LOCAL_EVENT == LOCAL_EVENT)\n",
    "    ).with_columns(\n",
    "        pl.col('local_ts').alias('ts')\n",
    "    ).group_by_dynamic(\n",
    "        'ts', every='1000000000i'\n",
    "    ).agg(\n",
    "        pl.col('exch_ts').last(),\n",
    "        pl.col('local_ts').last()\n",
    "    ).drop('ts')\n",
    "    \n",
    "    data = df.to_numpy(structured=True)\n",
    "\n",
    "    order_latency = np.zeros(len(data), dtype=[('req_ts', 'i8'), ('exch_ts', 'i8'), ('resp_ts', 'i8'), ('_padding', 'i8')])\n",
    "    generate_order_latency_nb(data, order_latency, mul_entry, offset_entry, mul_resp, offset_resp)\n",
    "\n",
    "    if output_file is not None:\n",
    "        np.savez_compressed(output_file, data=order_latency)\n",
    "\n",
    "    return order_latency"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "09772349-cebb-4698-8c6f-b77cbd19f15b",
   "metadata": {},
   "outputs": [],
   "source": [
    "order_latency = generate_order_latency('btcusdt_20200201.npz', output_file='feed_latency_20200201.npz', mul_entry=4, mul_resp=3)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.14"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
